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ABSTRACT 

The report of the Interagency Language Roundtable* s 
invitational symposium on language aptitude testing consists of a 
doscription of the project and appendixes which include the 
following: (l) the symposium program and abstracts of papers; (2) a 
list of participants; and (3) summaries of the discussions of three 
working groups (on applications, research, and instrumentation 
respectively) . The symposium was developed in response to intense 
interest in testing for foreign language learning aptitude among 
government agencies involved in foreign language training. It 
included individual and panel presentations by representatives of the 
National Cryptologic School, Federal Bureau of Investigation, Public 
Service Commission of Canada, Defense Language Institute, Central 
Intelligence Agency, National Security Agency, Army Research 
Institute, Foreign Service Institute, U.S. Air Force Academy and 
academic institutions. The presentations included these topics: 
apcitude tests currently in use in government agencies; cognitive 
abilities and foreign language aptitude; the uses and limitations of 
various tests in public agencies; predictors of success in intensive 
language learning; a new language aptitude test using an artificial 
language; learning styles, strategies, and aptitude; cognitive models 
of students* language structure based on intelligent 
computer-assisted instruction; the relationship between cognitive 
development and language proficiency; brain hemisphericity, language 
aptitude testing, and prediction of success in foreign language 
learning; and aspects of attitude, motivation, and personality in 
language learning, (MSE) 
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Accurate measuring of aptitude for learning modem foreign languages is of definite 
concern to many government agencies, including the Central Int^g^nce Agency (CIA), the 
National Securitj Agency (NSA), the Foreign Service InstitutfeT^""SI), the Federal Bureau 
of Investigation (FBI) and the Defense Lanniage Institute (DLI). In all these agencies, 
language aptitude testing plays an important role, most conu. only in selection for and/or 
placement in language training programs. Thus, it is not surprising that in the spring of 
1987, the Interagency Language Ro; idtable (ILR), which is composed of representatives 
of all the government agencies invoked or interested in foreign language training, discussed 
the need to pursue a major initiative in the testing of foreign language learning aptitude. 
As a result, a special section of the one day conference for govenmient linguists held prior 
to the 1987 Georgetown University Roundtable on Languages and Linguistics was devoted 
to language aptitude testing. Given the amount of interest demonstrated in the subject by 
those in attendance, the ILR Testing Committee subsequently voted to propose to the ILR 
Management Committee that a conference on language aptitude 'esearch and testing be 
held as soon as feasible. The Management Committee endorsed this conxpt and presented 
it to the full BLR body in May of 1987. 

In September, 1988, the ILR Invitational Symposium on Language Aptitude Testing 
became a reality. The two and one half day meeting was held on September 14-16, 1988, 
at the Foreign Service Institute Language School in Rosslyn, Virginia.' Funding for the 
symposium was provided by the following goverrment agencies: the Central Intelligence 
Agency (CIA), the Defense Language Institute (DLI), the Federal Bureau of Investigation 



Due to the unavailability of a large conference room at the FSI on September 14, 
meetings on that day were held at the nearby Westpark Hotel. 
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(FBI), the National Security Agency (NSA) and the U.S. Department of Education. 

During the planning of the conference, the Center for Applied Linf;uistics (CAL) was 
pleased to serve as "secretariat" for this important event under contract with the CIA. The 
organizational committee fo* the symposium consisted of Thomas Party (CIA), Madeline 
Ehrman (FSI), James Child (Department of Defense), Charles Stansfield (CAL) and Dorry 
Kenyon (CAL). This committee met mrjnthly from January to May, 1988. Activities 
involved in organizing the '•onference included choosing the dates and location, designing 
the structure of the symposium, ide.:tifying potential speal:ers from the academic community 
to prepare and present commissioned papers on important relevant topics, and identifying 
potential presenters from within the government agencies to present on current research 
within the government 

During the spring of 1988, the academic presenters named by the committee were 
personally contacted by Dr. Stansfield. Letters to poteniial government presenters named 
by the committee and letters soliciting other government presentations were also sent out 
by CAL at the same time. By the June ILR bimonthly meeting, the symposium program was 
completed and presented to the whole body for discussion. The final printed program is 
presented in Appendix A. 

Three goals guided the plaiming of the symposium. The first goal was to ensure that 
the meeting would infuse the ILR discussion of language aptitude testing with the latest 
research on relevant topics. Another goal was to foster cooperation among the agencies to 
the fullest extent possible. Lastfy, though perhaps most importantly, the symposium was to 
be work-oriented. One of the major corference goals was the production of a final written 
set of recommendations for future work in the area of Language Aptitude Testing. The 
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following discussion shows how these goals were met. 

In order to bring the latest research from the academic world into the government's 
discussion of language aptitude testing, three experts in the field were commissioned to write 
papers for presentation at the conference. Unfortunately, a fourth presenter, Maijorie 
Wesche o^ the Un^e^sity of Ottawa, was unable to participate. The three commissioned 
speakers and their topics were: 

1. John Carroll (University of North Carolina, emeritus) on ''Cognitive Abilities 
and Foreign Language Aptitude: Then and Now,'' 

2. Rebecca Oxford (Annenberf C Project) on "Styles, Strategies; and Aptitude: 
(mportant Connections for Language Learners," and 

3. Robert C Gardner (University of Western Ontario, Canada) on ^Attitude, 
Motivation and Personality as Predictors of Slxccss in Foreign Language 
Learning" 

Each of the above presentations was scheduled to last approximately 45 minutes, to 
be followed by an additional 45 minutes for questions and discussions. Each commissioned 
presenter received an honorarium plus reimbursement of expenses incurred in attending the 
symposium. 

In lieu of Maijorie Wesche's attendance, E Franklin Jacobus of Davidson College 
was accepted as a fourth presenter from outside the government. His paper was entitled 
"The Relationship between Cognitive Development and Foreign Language Proficiency." 

In order to serve the needs of the sponsoring agencies and to encourage cooperation 
among them, there were four special facets in the structure of the symposium. First, the 
symposium opened with a panel presentation of descriptions of the five foreign language 



aptitude tests currently in use at the various agencies. Second, on a second panel on the 
first day of the conference, representatives from the National Ciyptologic School, FSI, CIA, 
DU, and FBI discussed the role of language aptitude testing in those agencies-what is 
currently l^ing done and what problems and needs exist. This panel was joined by two 
representatives from the Public Service Commission of Canada, which has a very large and 
well-known language training program that makes extensive use of language aptitude tests. 
Third, there were six presentations by government employees reporting on research currently 
being undertaken in the government These presentations invoh^ed ten people and 
represented the following agencies: DU, OA, NSA, Army Research Iiistitute (ARI), FSI 
and the U.S. Air Force Academy, (For the names, titles and abstracts of all the 
presentations see the symposium program, Appendix A.) These presentations were planned 
to be approximately 30 minutes long, with at least 15 minutes of discussion following. 
Finally, the working groups described below allowed participants from various agencies to 
work closely together. To ensure that the conference would be work-oriented, three 
facets were designed into the program. First, the number of invited participants was limited 
to 65, During the summer, iiwitations were sent out from CAL to the various government 
agencies. Each agency was limited to a certain number of participants, k addition, an 
in^dtation to send one representative each was extended to both the Educational Testing 
Service (ETS) and the American Council on the Teaching of Foreign Languages (ACTFL). 
The actual number of participants was 58, (Appendix B contains a list of all participants 
and their addresses. This 1i!?t was sent to all participants following the symposium to 
encourage funher future cooperation among the agencies.) 



Second, copies of the presentation papers would be made available to all conference 
participants by the time of the conference. This goal was achieve ' with the exception of one 
government presentation. Presenters sent their papers to CAL. The four received by 
August 15 were reviewed by Dorry Kenyon and then revised by the author(s). Before the 
symposium the papers received were duplicated and then distributed to all participants at 
the symposium. Appendix C contains these papers in alphabetical order by &st author. 

Third, a special feature of the conference was time allotted for participants to divide 
into three smaller working groups. Each group had a facilitator and a specific focus: 
Language Aptitude Testing-Instrument Design, Research in Language Aptitude Testing, and 
Applications of Language i^titude Testing. These working groups met initially at the end 
of the first day of the symposium, and then for tvi'o hours on the last morning. Contrary to 
the schedule presented in the printed program (Appendix A), after the Friday morning 
coffee break all participants met together one last time. At this final meeting the facilitator 
from each group presented to the whole body a summary of the group's discussion. (The 
written summary from each of these groups is in Appendix D.) It wco cut of these working 
groups that the major goal of ti.e symposium was met, namely, to work towards writing long- 
range research agenda towards the development of more appropriate language aptitude 
measures than are currently available or in use. After the three working groups presented 
their summaries on Friday, it was decided by all present th.'u a position paper be written and 
presented to the ILR testing committee for discussioa and then to the main body of the 
ILR itself. 

In addition to the writing of a position paper, another important outcome of the 
symposium will be the published collection of the papers presented. This collection will be 
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printed in 1989 by Prentice Hall as part of the Language in Education series of the 
ERIC/acaringhouse on Languages and Linguistics. 

To ensure that the academic community is informed of the event, announcements 
about the symposium have been prepared and sent to the Modem language Journal. 
System, and Language Testing Vpdaj^. In addition, Charles Stansfield, Thomas Pany, 
Madeline Ehrman and Rebecca Oxford will be presenting a report on the symposium 
entitled Testing Foreign Language Aptitude in the United States Government" on 
November 19, 1988, at the annual ACTFL conference. The above, along with the set of 
recommendations to the ILR, will make sure that the symposium has an impact not only in 
the government, but outside the government as well. 

During the symposium, CAL staff (Charles Stansfield, Dony Kenyon, John Karl, 
Laurel Winston and Stephanie Kasubosld) made sure everything ran smoothly and that all 
the proceedings were correctly tape-recorded. Cheryl Francis of the FSI served as the on- 
site liaison. 
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PROGRAM 



Wednesday Morning, September 14 
Dogwood Room, Westpark Hotel 

8:30 - 8.40 Welcome 

Charles Stansfield (Conference Moderator) 

8:40-9:20 Panel Presentation 

Topic: Descriptions of Aptitude Tests Currently in Use 

MLAT Thea Bruhn 

ALAT James Child 

DLAB John Clark 

VORD James Child 

PLAB Charles Stansfield 

9:20 - 10:00 Academic Presentation 

John Carroll: Cogninve Abilities and Foreign Language Aptitude: Then 

and Now 

10:00- 10:45 Discussion of Dr. Carroll's presentation 

10:45- 11:00 Coffee Break 

11:00- 12:00 Panel Presentation 

Topic: The Uses/ limitations of Aptitude Testing in Various Agencies 

FSI The? Rruhn 

FBI Manjke Walker 

CIA Tliomas Parry 

DLI John Lett 

NCS James Child 

PSC/Canada Angie Todesco and Therese Castonguay 

12:00-1:15 Lunch Break 
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Wednesday Afternoon, September 14 

Dogwood Room, Westpark Hotel 



1:15-2:00 Continuation of 11:00 panel discussion. 



2:00-3:45 I .entations 



John Lett: Predictors of Success in an Intensive Language 

Frank E. O'Mara Learning Context: An Explanatory Model of Classroom 

Language Learning at the Defense Language Institute 



Discussion 



Thomas Parry: Preliminary Investigation of the Relationship 

J'^ 'S Child between VORD, MLAT and Language Proficiency 

Discussion 

3:45 - 3:55 Orientation to Working Group Objectives 

Charles Sta'nsfield 

3:55-4:15 Coffee Break 

Walk to the Foreign Service Institute 

4:15-5:00 Initial Working Group Meetings 

At the Foreign Service Institute 

Facilitator Room Wo rking Group Title 

Rebecca Oxford 212 Applications of Language Aptitude Testing 

Thomas Parry 213 Research in Language Aptitude Testing 

Charles Stansfield 214 Language Aptitude Testing-Instrument Design 

5:00-6:00 t^ocial Hour 

Meet at the Vantage Point Lounge, top of the Westpark Hotel 
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Thursday Morning, September 15 
Room 101, Foreign Service Institute 



8:30-9.15 



Academic Presentation 



Rebecca Oxford: 



Styles, Strategies and Aptitude: Important Connections for 
Language Learners 



9:15 - 10:00 



Discussion of Dr. Oxford's presentation 



10:00-10:15 



Coffee Break 



10:15-12:15 



Presentations 



Joseph Psotka: 
Merryanna Swartz 
Melissa Holland 
Seymour Hanfling 



Discussion 

Frank Jacobus: 

Discussion 
Betty Lou Leaver: 

Discussion 



Cognitive Models of Students' Language Structure: 
The View from Intelligent Computer Assisted Instruction 



The Relationship between Cognitive Development and 
Foreign Language Proficiency 



Brain Hemisphericity, Language Aptitude Testing and 
Prediction of Success in Foreign Language Learning 



12:15-1:30 



Lunch Break 
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Thursday Afternoon, September 15 
Room 101, Foreign Service Institute 

1:30-2:15 Academic Presentation 

Robert Gardner: Attitude, Motivation and Personality as Predictors of 

Success in Foreign Language Learning 

2:15-3:00 Discussion of Dr. Gardner's presentation 

3:00-3:15 Coffee Break 

3:15 - 5:00 Presentations 

Madeline Ehrman: Tke Role of Personality Type in Adult Lan guage Learning: 

An Ongoing Investigation 

Discussion 

Michael Bush: 772^ Personality and Attitudinal Dimensions of Language 

Aptitude 

Discussion 

5:00-6:00 Social Hour 

Meet at the Vantage Point Lounge, at the top of the Westpark Hotel 
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Friday Morning, September 16 

Foreign Service Institute 



8:30 - 10:00 

Facilitator 



Working Group Meetings 

Room Working Group Title 



Rebecca Oxford 211 
Thomas Parry 213 
Charles Stansfield 214 



Applications of Language Aptitude Testing 
Research in Language Aptitude Testing 
Language Aptitude Testing-Instrument Design 



10:00-10:15 



Coffee Break 
Room 101 



10:15 - 12:00 



Working Group Meetings (continued) 



ABSTRACTS 

(in alphabetical order by first presenter) 



The Personality and AttUudinal Dimensions of Language Aptitude 

Michael D. Bush 
United States Air Force Academy 

For some years educators and linguists alike have been engaged in an attempt to understand 
and thus explam the process of acquiring a second language. There exists on one hand the reality 
that every "normal" person is successful at learning his or her native language with little apparent 
dithculty. Yet on the other hand, there is no doubt in research on second language acquisition that 
abilities to learn a new language differ widely from person to person. This paper presents findings 
u Tt^c^ A ^^^^ ^^^^^ thousand pieces of data on approximately 1,200 students at 

the U. 5). Air Force Academy. The resulting scales and variables were formulated into a regression 
equation that considered various dimensions of personality, attitudes, and interests and looked at 
how well these help predict success in second-language acquisition. A model was formulated 
around the notion that success in the language learning endeavor is determined by aptitude and 
motivation with the first, factor being significantly influenced by "intelligence" and personality 
vanables. Attitudes and interests related most closely with the motivation factor 



Cognitive Abilities in Foreign Language Aptitude: Then and Now 

John B. Carroll 

University of North Carolina 

Although some skepticism is expressed regarding the possibilities of greatly improving 
cuirently available tests and procedures for predicting and diagnosing rates and degrees of success 
in foreign language learning, suggestions are offered on possible approaches: (1) revision of 
cuirently available tests to remedy minor defects, and construction and standardization of alternate 
forms; (2) extending currently available tests in the several domains of ability already known to 
have predictive value by adding tests to refine discriminations among those abilities, including 
further tests of auditory abilities; and (3) further study of the cognitive operations involved il 
foreign language learning, with attempts to develop tests and other procedures (e g , work sample 
tests) that would better capture the essences of these cognitive operations 



The Role of Personality Type in Adult Language Learning: 
An Ongoing Investigation 

Madeline E. Ehrman 

Foreign Service Institute 

This paper describes an ongoing project to examine the utility of Carl Jung^s typology of 
conscious functioning in an intensive language training setting. A psychological instrument, the 
Myers-Bri jgs Type Indicator (MBTI), is built on the psychological type model, and through this 
instrument, J am investigating the interaction of student learning styles, preferred learning 
strategies, and specific external variables. 

This paper discusses some questions about the meaning of language learning aptitude, 
describes the MBTI model and the research project, presents some of the findings that have 
emerged so far, and addresses the potential relevance of personality type to the concept and 
measurement of language aptitude. 

Because quantitative findings are being published elsewhere, the present paper only 
summarizes them and focuses primarily on preliminary qualitative results. These are sketched in 
the light of interaction of methodology and type, insights into the nature and behavior of FSI 
students, an analysis of teacher and training supervisor interaction, some cross cultural 
applications, and personal and professional impact on the author. 



Attitudes, Motivation and Personality as ^redictors 
of Success in Foreign Language Learning 

Robert C. Gardner 

University of Western Ontario 

The purpose of this paper is to review literature concerned with the relation of two classes 
of variables, attitudes and motivation, on the one hand, and personality characteristics, on the 
other, to achievement in a second language. Based on this review, it is concluded that the evidence 
is not strong for many relationships between personality traits and second language acquisition, 
whereas it is reasonably clear that attitudinal/motivational characteristics are involved. In 
considering the poor results of possible personality correlates, it is speculated that this could be due 
to the fact that researchers do not construct their measurement instruments of personality correlates 
to include the language learning context, and the trait of anxiety is used as a relevant example. That 
is, whereas anxiety is not a consistent predictor of proficiency in a second language, language 
classroom anxiety and language use anxiety are. It is proposed that the reason why 
attitudinal/motivational variables are better predictors of proficiency is because by and large 
researchers have tended to focus on attitudes and motivation that are directly relevant to the 
language learning context. 

Research concerned with attitudes and motivation is then summarized, and it is 
de-^onstrated that attitudes and motivation form one class of variable that is relatively independent 
of language aptitude. Both are shown to be relatively consistent correlates of second language 
proficiency and that, combined, they offer good prediction of achievement. Related results 
demonstrate that, although both language aptitude and attitudinal/motivational indices demonstrate 
such 'convergent validity, only the attitudinal/motivational indices have discriminant validity. 
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In discussing this research, a distinction is made between common and idiosyncratic 
"causes" of second language achievement, and it is emphasized that, since research can only deal 
with the common causes, prediction will always be somewhat restricted. Attention is also directed 
toward a number of measurement and analytic problems that must be considered primarily when 
dealing with the relation of personality and attitudinal/motivatlonal characteristics to second 
language achievement. These include such things as confounding levels of proficiency and/or 
course-related characteristics by merging classes, developing measures of variables that confound 
the attribute in question with second language achievement, etc. The relevance of this to the 
prediction of success in second language learning by adults is considered. 



The Relationship between Cognitive Development 
and Foreign Language Proficiency 

E. Franklin Jacobus, Jr. 

Davidson College 

This paper hypothesizes a link between cognitive development and aptitude for attaining 
levels of communicative proficiency in a foreign language. The hypothesis is based on empirical 
data. 

In William G. Perry's theory, adults organize their view of the nature and acquisition of 
knowledge or truth in hierarchical, sequential stages called Dualism, Multiplicity, Relativism, and 
Dialectic. In broad terms, individuals in Dualism view truth as black or white and authorities as its 
source. Individuals in Multiplicity 'lew truth as black and white but difficult to discover because 
there exist multiple truths from different authorities; personal whim is a means of choosing truth. 
Individuals in Relativism view truth as a conclusion reached from within a particular perspective 
and acknowledge that different perspectives and truths are possible. Individuals in Dialectic are 
able to commit themselves to one truth among perspectives but view it as open to modification 
from new thought and data. 

My research suggests that there is a correlation (r=.60) between Perry level and the 
attainment of levels of language proficiency as measured by the ACTFL Oral Interview. I draw 
two inferences. First, to the extent that learning a language has hierarchical communicative goals, 
Perry intellectual level may put a limiting factor on the ultimate level of communicarive proficiency. 
Second, Perry intellectual level may affect the speed and ease of language learning since the stages 
may shape the learning strategies chosen and the affective attitude felt in foreign language 
acquisition. 
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Brain Hemisphericity^ Language Aptitude Testings and 
Prediction of Success in Foreign Language Learning 

Betty Lou Leaver 

Foreign Service Institute 

This presentation analyzes data from a 4-year study on the interrelationship of patterns of 
brain hemisphere dominance, MLAT scores (and in some cases, DLAB scores), and the rate of 
success in language acquisition for students of Russian studying in a ten-month intensive program 
designed to deliver students to the S-3/R-3 proficiency levels. Results of the study indicate that, at 
least for English-speaking students of Russian, the MLAT is an adequate criterion of ultimate 
success in language learning only for students displaying integrated brain dominance and is less 
than adequate for students displaying either right-bram or left-brain dominance (for different 
reasons in each case). In the case of right-hemisphere dominant students, the DLAB scores appear 
to be a better predictor of success, particularly in those cases in which the DLAE and MLAT scores 
differ significantly. Brief references will be made to differences in learning styles measured on the 
Kolb and Gregorc scales which may influence overall patterns of success in language learning. In 
all cases where only one measurement instrument was taken into consideration, the most clear-cut 
differences in rate of language acquisition and relative strength in subskill areas were best predicted 
by tests of hemisphericity. 



Predictors of Success in an Intensive Language Learning Context: 
An Explanatory Model of Classroom Language Learning at the 
Defense Language Institute 

John A. Lett, Jr. 

Defense Language Institute 

Frank F. O'Mara 

Advanced Technology, Inc. 

Cince February 1986, the Defense Language Institute and the Army Research Institute have 
been collecting predictor and proficiency data frorn 1900 selected students in DLFs basic language 
courses in Korean, Russian, German, and Spanish. The results of preliminary analyses of the 
extensive data set will be presented orally and via overhead transparencies in this session. 
Criterion measures are scores on the Oral Proficiency Interview and the Defense Language 
Proficiency Tests; predictor measures include data on an array of individual characteristics, 
including (but not limited to) scores on an adaption of Robert Gardner's attitude/motivation battery, 
a learner strategies inventory, and a variety of cognitive style instruments. The results of a variety 
of multivariate analyses will be presented, and their implications will be discussed, including the 
tenability of the Gardner model and others when applied to language learning in the DLI context. 
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Styles, Strategies and Aptitude: Important Connections for 

Language Learners 



Rebecca Oxford 

Annenberg/CPB Prcjcct 



The topic of language learning styles and strategies has recently come, into vogue among 
researchers and practioners. However, few empirical studies or synthesis papers have attempted to 
ink styles and strategies, much less to associate these two concepts with a third phenomenon 
language learning aptitude. This paper summarizes research results currently available on these 
topics and explores their possible interconnections. Specific implications for language aotitude 
testing are presented. & & f » 



Preliminary Investigation of the Relationship between VORD, MLAT, 

and Language Proficiency 

Thomas S. Par.y 

Central Intelligence Agency 

James Child 

Department of Defense 

J ^^W^^if ''^'^ P^P^^ '° ^^P°" preliminary findings of a joint exploratory study 
conducted in 1987-88 between the Department of Defense and the Central Intelligence Agency to 
shed further light on the the psychometric properties of VORD, a new language aptitude test that 
uses an artificial language. Based on Turkic language structural typologies, VORD was developed 
m response to a need for an instrument that could better predict success in learning non-Indo- 
European languages. Such commonly used language aptitude tests as the Army Language Aptitude 
Test (ALAT) and the Modern Language Aptitude Test (MLAT) were developed and validated in the 
late 1950 s to predict learner success particularly in Western European languages. The present 
study examines intercorrelations between VORD and MLAT subtests and the reliability of each 
measure in predicting learner end-of-training language proficiency outcomes. 

The smdy was carried out in two phases. In phase one, the following research question 
was posed: Do significant coirelations exist between MLAT and VORD subtests'^ From this 
question, two tangentially related questions were posed: 1) Is there a significant correlation 
between performances on MLATA'ORD and learners' perceived aptitude to learn foreign 
languages?, and 2) Is there a relationship between aptitude test performance and such learner 
vanables as time-in-training, age, gender, level of motivation, and overall satisfaction with 
language training? For phase two of the investigation, the following questions were posed- 1) 
Do significant coirelations exist between learner per.brmance on MLATA^ORD and outcomes on 
^/"^n°i"^^'"^"^ reading proficiency tests?, and 2) Which subtests of the MLAT and 

VORD, either individually or in combination, are the strongest predictors of oral and reading 
proficiency test outcomes? 

Thirty-six subjects (17 male and 19 female) enrolled in a government language training 
program volunteered to participate in the study. All were native speakers of English ranging in age 
frorn 21 to 56. Many had completed several years of service abroE-' and were learning their second 
and in some cases their third language. Subjects completed the VORD, MLAT, a questionnaire, 
and end-of-training oral and reading proficiency tests. Data were analyzed using the standard 
correlation and regression programs of the Statistical Analysis System (S AS). 



ERIC 



20 

10 



For phase one of the study, data analysis revealed that significant moderate correlations 
exist between MLAT and VORD composite scores (r=.695, p<.01) and that correlations between 
MLAT and VORD subtests ranged from low to moderate (r= 20 to .68) with the moderate 
correlations being significant (p<.Ol). On the question of learners* perceived aptitude, subjects 
viewed themselves as average to slightly above-average language learners on a range from poor to 
superior. They also tended to score in the average range on the MLAT (per government norms). 
The correlation between these variables was found to be moderate and significant (r=.727, 
p<.00l). Subjects scored in the 43rd percentile on the VORD (no norms), resulting in a mild but 
significant correlation with learner perceived aptitude (r=.450, p<.05). No significant correlations 
were found to exist between the variables of age, level of motivation, overall satisfaction with 
language training and MLATA^ORD subtest and composite scores. The time-in-training did not 
correlate with any of the VORD subtests, but was found to be significantly correlated with MLAT 
composite scores (r=.591, p<.05) and MLAT subtest III (r=.621, p<.Ol). 

Data analysis for phase two revealed mild correlations between performance on the MLAT 
(composite scores) and speaking proficiency (r=.476, p<.Ol) and reading proficiency (r=.446, 
p<.Ol). Mild correlations were also detected between VORD composite scores and speaking 
proficiency (r=.463, p<.Ol) and reading proficiency (r=.345, p<.05). Combining VORD/MLAT 
subtests in a stepwise regression analysis, MLAT subtest II, phonetic script, was found to be the 
strongest predictor of reading proficiency while MLAT subtest III, spelling clues, was the 
strongest predictor of speaking proficiency. Of the four VORD subtests, the sentences subtest 
proved to be the strongest predictor of both language skills. MLAT composite scores were 
significantly better overall predictors of both speaking and reading language proficiency than 
VORD composite scores. 

Although limited by small sample size, the present study provides evidence that MLAT is 
more effective than VORD as a predictor of language proficiency outcomes. There is evidence, 
however, that VORD may be a better predictor of learner outcomes in carrying out such discrete 
language tasks as grammatical analysis. This leads the researchers to conclude that language 
aptitude is more than a unidimensional construct 



Cognitive Models of Students^ Language Structure: The View from Intelligent 

Computer Assisted Instruction 

Joseph Psotka 
Merryanna L. Swartz 
Melissa Holland 
Seymour Hanfling 
U. S. Army Research Inslilule 

Our work is aimed at developing job and training aids for military Intelligent Computer- 
Assisted Insti'uction (ICAI) foreign language training. Two technologies that show great promise 
for improving training are hypertext and natural language processing. Hypertext provides a text 
based system that goes beyond text to include graphics, video, and sound (hypermedia) as well as 
links, cross-references, and network or lattice strictures. Natural language processing refers to the 
use of parsers, grammars, and dictionaries to provide computer-based language fac?'nies. In order 
to individualize instruction to make it as effecilve as one-on-one tutoring, we must c eate effective 
student models.that capture students* knowledge structures and skills. Our goal, therefore, is to 
investigate these technologies to see how they may best be used to articulate student knowledge 
structures. In particular, we are interested in the web of links and nodes that relate students* 
knowledge in one language with their knowledge of similar semantic and syntactic structures in the 
language they are learning. 
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The specific focus of o'lr project is the application of these technologies to the teaching of 
second languages in the schoolhouse. The principle components are: use of a hypertext, 
multilingual authoring environment for language instruction and the presentation of fundamental 
pnnciples, vocabulary, and grammar; and use of an interactive parser and grammar for on-line 
translation, error-correction, and remediation of translation exercises and conversation. Both of 
these components are invested heavily by the technologies of Artificial Intelligence: Intelligent 
Computer Assisted Instruction, Machine Translation and Natural Language Processing. 

We are investigating ways to improve upon these technologies to increase the effectiveness 
of training for the acquisition, sustainment, and automatic assessment of foreign language skills. 
Particularly, we are implementing techniques for modelling the cognitive skills underlying foreign 
language competence using computational linguistic models and semantic networks built in 
hypertext systems. These techniques may prove useful for assessing basic competencies. 
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SUMMARY OF APPLICATIONS GROUP DISCUSSION 
AT THE INTERAGENCY LANGUAGE ROUNDTABLE INVITATIONAL SYMPOSIUM 

ON LANGUAGE APTITUDE TESTING 



Comnents Sunnarlzed by Rebecca Oxford, Annenberg/CPB 

Group Members: Marljke Walkr>r, Olga Navarrete, 

Bryce Chrlstensen, FBI 
Betty Leaver, Madeline Ehrman, FSI 
Th^rese Castonguay, PSC 
Rebecca Oxford, Annenberg/CPB 

Assistants: Stephanie Kasuboskl, Laurel Winston, CAL 



PURPOSES FOR ASSESSING PREDICTORS OF LANGUAGE LEARNING SUCCESS 

This group Identified the following purposes for assessing 
predictors of language learning success: 

1. Selection (In or out) - current programs 

2. Qualification for future programs 

3. Placement/streaming Into different kinds of programs 

4 . Tailoring/ Individualization 

5. Dlagnosls/coiinsellng 

TERMINOLOGY 

The group iinanlmously decided It would be helpful to abandon the 
term "aptitude testing," because of Its restrlctlvely cognitive 
connotations. Preferred terms Included: 

1. AssMslng the predictors of language learning success 

2. Assessing the enhancers of language learning success 

These were viewed as more comprehensive terms which could Include 
affective, social, style, and strategy variables as well as 
cognitive factors. 

DEFINITION OF LANGUAGE LEARNING SUCCESS 

Lancmaqe learning success Is likely to be viewed differently by 
different agencies. Some agencies would consider the learner 
successful if proficiency is shown in all four skills; other 
agencies are concerned only with the receptive skills. 

In addition, language learnin g success will also be differently 
defined for various professions . The constant example is the 
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translation profession, which requires very different skills from 
many other language professions. 

The definition of language learning success must be carefully 
considered before it is possible to choose relevant predictors of 
language learning success. 

THE NEED FOR A BATTERY OF PREDICTIVE INSTRUMENTS 
AND A PROFILE 

The group felt that a single test score is not useful. We need a 
mul t idimens ional profile arising from a battery of predictive 
instruments . Candidate variables include: 

1. Cognitive/linguistic (standard aptitude-type tests) 

2. Attitude/motivation related to language learning, 
target culture, etc. 

3. Personality variables as relevant 

4. Learning styles (e.g., brain henisphericity, 
global/analytic, MBTI-type) 

5. Learning strategies 

6. Background, learning experiences, etc. 

It was suggested that a personal interview be used to validate 
results of the more objectively developed profile. Host agencies 
are already doing intensive interviews, and their representatives 
felt that in most cases the interview data could be made 
available. 

CONSTRUCTION OF THE BATTERY: 

The following Meas are not in any particular order, nor do they 
represent all the steps in the construction of the battery. They 
do offer some useful suggestions, however. 

1. Include a fakability scale (social desirability) as part of 
one of the instruments. 

2. Use the best of the existing aptitude subtests to measure 
the cognitive predictors of language learning success. 

3. Mix easy and hard items if possible on a speeded test to 
enable reflectives and impulsives to reveal themselves. 

4. Use a scoring system that checks for speed and accuracy 
separately. 

5. Include some power (tmspeeded) testing. 

6. Include some items that predict differentially for different 
language families (e.g., Romance, Slavic, etc.) 

7. Be sure to include dimensions beyond the cognitive (e.g., 
the dimensions listed abova under THE NEED FOR A BATTERY OF 

PREDICTIVE INSTRUMENTS AND A PROFILE) . 

POLITICAL ISSUES 

1. Don't use the term "personality" in some agencies. 

2. Focus on the need for a sufficiently long testing peiriod. 
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Justify through cost-effectiveness and cost-benefit figures. 
(Ex.t it costs $75K for 1 student for 1 year at DLI, 
according to one estimate; need to predict success 
accurately, or else this investment is lost through 
the individual's failure and dropping out./ 
3. The better our prediction of language learning success 

factors, the more we will realize that one of these factors 
is outside the student's control: the instructional methods 
used. How much can we influence this factor? 

STREAMLINING THE LEARNING PROCESS VIA STRATEGIES 

He have all heard Dr. Carroll's statement that anyone can learn 
with sufficient time. But as Dr. Carroll also states, and as 
most of us agree, we can streamline the learning process (i.e., 
the amount of time and effort it takes to learn) by training 
students to use appropriate strategies. 

These strategies make learning more efficient and effective. 

Example: Netacognitive strategies for planning, self-evaluating, 
etc.; cognitive strategies for reasoning, analyzing, etc. 

Strategies can also help leaimers overcome a number of affective 
and social barriers. Example : Some agencies train their 
employees to consider certain countries as enemies, but then tliey 
expect the employees to want to learn the language of t-^ose 
countries; this sets up many affective and social barriers which 
have to be broken down. Strategies (such as consciously focusing 
on "good" aspects of the culture, centering on the language as a 
means of communication, pairing with a sympathetic native 
speaker, using positive self-talk, etc.) can overcome these 
barriers. 

ADAPTATIONS NEEDED 

Three kinds of adaptations seem to be needed: 

1. Train students to adapt themselves to an extent - through 
student training in learning strategies 

2. Train teachers to adapt themselves and the program to an 
extent - through teacher training in instructional 
strategies 

3. Train students to adapt the program materials to an 
extent - through student training in materials adaptation 
and especially through training in metacognitive strategies 
(so students will know how to assess their own 

learning needs) 

The point of these adaptations is CONVERGENCE of student, 
teacher, language program, and agency needs. 

Note that assessment and diagnosis are necessary parts of any 
adaptation. You have to know where you are in order to know 
where you're going 1 



NEEDED RESEARCH 

The following non<-prloritized areas need research: 

1. Plateauing - causes of (affective, cognitive, etc.)# fixes 
for 

2. Different styles and strategies most useful for lover levels 
and higher levels of language learning 

3. Differential prediction by language family, profession, etc. 

4. Reliability and validity issues 

5. Strategy training - what kind of training is optimal for 
whom? What are the effects? 
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Final Repoz^ of the Research Working Group 
of the Interagency Language Round Table 
Invitational Synposiun on Language 
Aptitude Asses ssent 

September 14-16, 1988 

SubBitted by Thomas S. Parry, Committee Chair 



The agenda for the Research Working Group included four 
major components. They were: 

1. Review and discuss crucia} research questions and 
issues with an eye toward prioritization. 

2. Develop a plan to conduct a government-wide needs 
analysis to determine what various agencies want to 
accomplish. 

3. Discuss possible funding sources for research projects. 

4. Develop a re ""istic timeline. 

Research Questions and issues 

The Research Working Group agreed that future research 
should continue to address the "non-cognitive" areas such as 
brain hemisphericity, learning/teaching styles and strategies, 
attitude, and motivation. Little is really known about brain 
hemisphericity. It is an area of tremendous relevance in better 
understanding the language learning/acquisition process. Recent 
research efforts in this area have been fruitful and have 
provided an excellent foundation for continued work in the years 
ahead. Further research in the area of learning/teaching styles 
and strategies as well as attitudes and motivation must address 
refinements in instrument design, construct validity, 
clarification of operational definitions, and utilize research 
designs employing an aptitude-by-treatment-interaction (ATI) 
framework. 

From a criterion standpoint, future research efforts should 
focus on the multidimensional nature of language proficiency and 
how to predict success in mastering specific language skills 
(i.e. speaking, reading, listening, and writing) at different ILR 
levels of proficiency in specific languages. Because proficiency 
sets forth the framework for most language instruction and 
testing within the U.S. government, refinement of the guidelines 
for the receptive and writing skills at each ILR level would 
appear to be a priority. 

Specifically, the questions that need to be addressed by 
each USG agency are: 1) What language or languages should we 
teach? 2) On which of the four skills should we focus our 
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instruction? 3) To what level (s) of proficiency shall we teach? 
and 4) What applications of language skill (s) will need to be 
taught (i.e. general, job-related, or job-specific)? Knowing the 
answers to these questions will aid USG research efforts to 
develop useful measures to predict success in these areas. 

Finally, the issue of whether there is a ""translation 
aptitude** was discussed. Discussion focused on two luiajor 
questions: Is there such a thing as translation aptitude and if 
so, how would it be measured? Translation rec[uires a highly 
specialized verbal ability to parse very complex structures in 
two or more languages. It seems that a measure of translation 
aptitude would need to assess an individual's sensitivity to do 
complex parsing similar to the ""Translator Readinesr Test"' used 
at the Department of Defense. 

Government-Wide Plan to Conduct Needs Analysis 

The plan to conduct a government-wide needs analysis 
included five steps. 

1. Discussion should begin within ILR Testing 
Committee concerning the desired outcomes of 
language training efforts. (In other words, what are 
the desired criterion outcomes?) 

2. Discussion should branch out from the ILR Testing 
Committee to include management personnel of the 
respective agencies in an effort to better understand 
their views on language training in general and 

language training outcomes in particular. 

3. Discussion should then be extended to include clientele 
from the various agencies who will be the recipients of 
language training. We need to know their goals, how 
they plan to use a foreign language in their various 
assignments and to what degree of proficiency. 

4. Inventory existing databases in various agencies to 
determine what can be analyzed and how it should be 
analyzed based on agency needs and prioritized research 
questions. 

5. Develop a marketing strategy, based on prioritized 
research questions, that aligns with various agencies' 
short- and long-term goals with an eye to selling the 
idea of ""ongoing research"" to senior management. 

Funding Sources for Research 

The group concensus was that it was too early to approach 
any agency or private foundation on the subject of funding 
without having a specific research plan or projects to propose. 
Possible sources to approach for future funding might include the 
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Department of Education (DOE) , The National Science Foundation 
(NSF) , and the National Endowment for the Humanities (NEH) • 

Timeline 

To accomplish all that has been set forth in this and other 
committee deliberations and reports may be possible in one 
calendar year. Considering workloads and priorities, however, 
eighteen to twenty- four months may be more realistic. The danger 
of carrying this preliminary groundwork too far into the future 
is that people and agencies will become disinterested and bored. 
For this reason, the development of the long-term research plan 
should reside with the ILR Testing Committee (as a focal point) 
and remain a high priority on the committee's agenda. 
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Final Report of the 
Working Group on Instrumentation 

Charles H. Stansfield 
Facilitator 

Members of this working group were: Nazih Daher (PSI) , 
Stephen Soudakoff (DLI/NCS) , Robert c. Gardner (University of 
Western Ontario), Pardee Lowe, Jr. (CIA), John L.D. Clark (DLI) , 
Dorry Mann Kenyon (CAL) and Charles w. Stanslield (CAL) , who 
served as facilitator. 

This report is divided into three parts. The first section 
describes the working group's outline of a proposed project for 
future work. The second section proposes specific steps that 
will need to be implemented in order to begin the project. The 
last section lists outstanding issues and research questions 
brought up in the working group's discussion but not directly 
incorporated into the proposed agenda. 

PROJECT OUTLINE 

An Interagency Project. The working group envisages an 
interagency project that would involve a number of governmental 
agencies in a number of different ways, including the 
contribution of personnel and funding. The medium for working 
together would be the ILR, which would provide the necessary 
framework. 

Focus on Foreign I^ncmaae Learning P r ognosis. The scope of 
the project would be broader than language aptitude testing per 
se. It is thought that foreign language learning prognosis would 
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better describe the goals of the project. That is, the goal of 
the project would be to iaprove the prediction of foreign 
language leerning success through a battery of measures that 
would not only include measures of cognitive variables, but also 
measures of attitude/motivation, personality, and perhaps 
learning strategies (as and if appropriate) . 

Coaputer-Based Testing. At the heart of the project would 
be computer-based testing. At the very least the project would 
incorporate computer-administered testing. If and when possible, 
computer adaptive testing techniques could be used. The 
technology of BIDS (Electronic Information Delivery System) would 
be used, making possible the use of audio and video components in 
the test administration. The system and the database that would 
drive the computerized testing program would be flexible enough 
to be adapted to the specific aeeds of the different agencies. 
The system would also be able to be adapted on an individual 
basis, or by any combination of agency needs and individual 
strengths in terms of specific jobs in specific agencies. 

Thrs& Variable Domains. Three interacting domains of 

variables would need to be considered. The first one would be 
student variables, including student background variables such as 
previous language learning exposure, length of any previous 
language training and proficiency level achieved. It was pointed 
out in the course of the working group's discussion that previous 
exposure to language learning is not an automatic positive 
predictor of foreign language learning success. For instance, 
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the fact that an individual after lengthy training reacheu only a 
low proficiency level in an "easy" language may be prove to be a 
negative predictor. The second domain of variables to be 
considered would be instructional variables, not only in terms of 
the goals of the instruction, but also in terms of the type of 
instruction under consideration. For exeunple, is placement in an 
analytically-oriented or proficiency-based program being 
considered? There are differences in methods and teachers among 
the various agencies that would need to he considered. Finally, 
the outcome criteria variables play a crucial role. These would 
include, as necessary, the level <.f proficiency to be reached 
within a certain tine frame, the skills to be mastered, and the 
specific language to be learned. 

TasH-Pftg^ Exercises. The working group felt that 

assessment would best be based on some type of work sample 
because, among other reasons, task-based exercises have a greater 
degree of face validity. The tasks wovld vary from language to 
language, from skill to skill, and with the needs of each agency. 
Examples of these variations include the need to assess the 
ability of a student to deal with a tonal language (e.g. 
Chinese) , in which case »n auditory task ^ay be required, and the 
need to assess the ability of a student to deal with a 
complicated %nriting system (e.g. Japanese), in which case a 
visual task may be required to assess sensitivity to writing 
form. 
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Test General and specific Aptitudes. The conputer based 
testing approach would allow the flexibility for the assessment 
to begin with a general aptitude neasure. Then, once there was a 
fix on general aptitude, the system would measure specific skill- 
by-language aptitudes. The approach vould allow for a battery of 
various subtests and a large item bank, to which various agencies 
could contribute. The system would be driven by the aptitudes 
needed by a specific agency and for specific positions within 
that agency, when desired. This would lead to the most optimal 
placement given the agency's needs. The computer-based approach, 
in which the examinee's responses would be el-sctronically stored 
by the computer, would enable a huge database to be built up. 
This database could be accessed for research purposes both by 
researchers inside the government and by those in the academic 
community when possible. 

STEPS TOWARD PROJECT IMPLEMENTATION 

The working group proposed t^e following specific steps 
towards implementation of this project. 

1. Assuming interagency cooperation, we would begin with a 
survey of the needs within each agency. This survey would 
identify needs in term of the languages taught, the use of 
language learning prognosis data (i.e. what application each 
agency makes of language learning prognosis information, 
whether for selection, placement, or diagnosis) and the 
foreign language skills needed in each agency. It was 
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envisioned that the data collected by the operational 
conputer-based testing project could be used for a variety 
of purposes for which scores on current aptitufe measures 
are not generally used. One of these uses could be in 
providing instructors with feedback eneibling them to enhance 
instruction, especially in reference to the teaching and use 
of individual strategies for both language learning during 
training and maintaining language skills after training. 

We would collect available measurement instruments, not only 
of aptitude, but also of attitude, motivation, learning 
styles, etc. 

Before writing task-based items, we would develop a taconomy 
of specific language-by-skill ^asks on an interagency basit;. 
This would be a taxonomy that identifies abilities that vary 
by skill crossed with language, such as the auditory ability 
to deal with tones or the visual ability to deal with a non- 
Roman alphabet. This taxonomy would addreas the question of 
what factors make one language harder than another for 
American Engl ish speakers to learn , thus enabl ing us to 
specifically develop measurement instruments uo discriminate 
among those abilities which best predict aptitude to handle 
those factors. 



4. The next step would be to compile, adapt and write items for 
the task-based exercises. This, too, would be on an 
interagency, cooperative basis. 

5. Concurrently with the above, we would need to examine the 
technology that would be required to implement this system. 
We would need to propose a plan of the hardware and software 
requirements of the project. 

QUESTIONS FOR FURTHER RESEARCH 

Paralleling the above proposed interagency project, the 
working group disqussed the following additional issues and 
research questions. 

1. Is there any room for a "cando" self -assessment (self- 
rating) in a battery of predictors of foreign language 
learning success? 

2. Nhat, if any, is the relationship of paper and pencil tests 
to predicting oral proficiency? 

3. How can aptitude for translation and interpretation be 
tested? 

4. What is the difference in testing for language aptitude for 
people who have had no foreign language learning experience 
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versus those who have? In other vords, what is the effect 
of prior language learning on the validity and predictive 
results of a language aptitude measure? How night these 
differences affect validity when using a single instrument? 

5. Is there a difference in predicting success in initial 
learning of a language versus testing for ultimate success 
at high levels of proficiency? 

6. How do available language aptitude tests perform in 
predicting for success in leaminc different languages 
according to a taxonomy of what is required to learn those 
languages by skills? 

7. What is the best type of score to be reported to managers — a 
single score or a composite? 

3. What is the most effective way to mask 
attitudinal/motivational questions so that exzuninees cannot 
"fake" their answers to them? 

9. How can we improve differential prediction of success for 
different languages or language families? 
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10. How can we best predict long-range language learning after 
completion of a training program and an individual's success 
in maintaining foreign language skills? 

i 
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